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ABSTRACT 

Provided is the description of a program planning and 
assessment process model which is technically based on multi^-variate 
analysis and can be used to evaluate special education programs. The 
model is noted to be divided into four sections: (1) specification of 
program objectives, (2) pre-assessment procedures, (3) evaluation 
methodology, and (4) post-assessment procedures. It is concluded that 
such a system vould provide the necessary information required to 
appropriately plan and assess special education programs. 
(Author/IH) 
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Abstract 



Traditional methods of program planning and assessment have failed to provide 
special education decision-makers with the necessary Information to measure 
the impact of innovations In their operating systems. Simllarlly evaluation 
studies In special education have the common goal of producing evaluative 
statements concerning the worth of particular educational activities. Most 
of these studies are lid t pursued by means of well-established, orderly, and 
effective procedures and the techniques Involved are often so primitive that 
they can hardly be described as techniques at all. 
f 

Currently the status of special education decision-making necessitates 
modem theory and methodology to provide for a program planning and assess- 
ment process which is systematically structured into a feasable operating 
system. Upon that basis, an attempt has been made to construct and illustrate 
the operations of a model which is technically based on multi-variate analysis 
The model has been divided into four sections: (1) Specification of Program 
Objectives; (2) Pre Assessment Procedures; (3) Evaluation Methodology; 
and (4) Post Assessment Procedures. Such a system would provide the necessary 
information required to appropriately plan and assess special education 
programs. 



Traditional methods of planning and evaluation have failed special educators 
In their attempts to assess the Impact of Innovations In operating systems 
(Guba^ 1969). Evaluation stxidles In special education have the common goal 
of producing evaluative statements concerning the worth of particular educa- 
tional activities. Most of these studies are not pursued by means of well- 
established, orderly and effective procedures, and Indeed, the techniques 
Involved are often so primitive that they can hardly be described as tech- 
niques at all (Coleman, 1966). Guba summarizes the current status of 
evaluation In special education on the note that what Is needed Is a tech- 
nology of evaluation. Upon that basis an attempt will be made to devise 
and demonstrate the usefulness of a multl-varlate planning and evaluation 
model. The development of the model will be around a core of elements 
necessary in any evaluation design. 

The model td.ll be used to assist special educators in the program planning 
and evaluation process. 

A composite model of planning and evaluation \d.ll be designed within the 
following general structure: 
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The mechanical procedures Involved In program planning and evaluation have 
been widely disseminated since they were first outlined by Ralph Tyler In 
a series of papers published over thirty years ago (1934). Central to such 
a program development process Is the specification of objectives In terms 
of so-called specific behaviors and the means of obtaining evidence of the 
extent to which particular objectives are achieved by a program utilizing 
a specified population of students (mentally retarded, learning disabled, 
emotionally disturbed, etc.). Since his early delineation of the mech- 
anical aspects of a systematic program development process, Tyler (1966) 
has specified the goals of educational evaluation as follows: 

• • • to appraise the achievement of Individual students; 

• • • to provide Information helpful In planning subsequent teaching; 

• • • to appraise the educational effectiveness of a curriculum. Inst- 

ructional materials, and procedures of administrative organiz- 
ational arrangements; and 

• • • to assess the educational progress of larger populations In 

order to provide the public with dependable Information to 
help In the understanding of educational problems. 

Central to the theme of this presentation Is the development of a multi- 
variate approach to systematic special education program planning and 
assessment. The point of view taken by this author Is that evaluation 
and assessment are to be viewed as Inter changable. And to briefly des- 
cribe the context of systematic program planning and development. It Is 
important to acknowledge that In the practical sense of operations, 
evaluation Is a matter of technique. An event occurs and the evaluator 
makes some record of the event. Careful study of the record generates 
factual statements serviceable as evidence for Inferences leading to 



generalizations, explanations. Interpretations, predictions, and decisions. 
In the theoretical sense of reasoning, evaluation Is a matter of using 
concepts, conceptual systems, constructs, models and theories. An event 
can be studied effectively If the evaluator generates a telling question. 
Such questions require key concepts or generative Ideas which lead and 
guide techniques (Kaplan, 1964). Thus, evaluation can be thought of as 
bringing together conceptual systems at the theoretical level, and tech- 
niques at the operations level. 

If one looks to the literature In the behavioral sciences (I.e., special 
education, psychology, sociology, education, economics, etc.). It Is 
apparent that many factors can account for 'the theoretical relationships 
underlying the practical acts of doing' observed as dally happenings In 
the universe of events. To ask or answer questions Involving complex 
relationships and numerous factors necessitates a device that allows the 
behavioral science Investigator to ask a multi-dimensional question. If, 
as is often done, one focuses upon a specific relationship, it is apparent 
that many of the variables which are theoretically relevant to the investi- 
gator are randomized while only one or two factors are examined. While it 
seems that we recognize complexities regarding assessed human behavior we 
often fall to conduct studies which reflect these complexities (Kelly, 1969). 

It is the position of this author that many special educators limit their 
assessment capabilities because the evaluation methods to which they have 
been exposed are limiting (Jones and Sommers, 1975 and 1976). A multl-varlate 
procedure such as multiple regression analysis has been stimulated by the 
invention and wj,de spread use of the high speed computer. Subsequently, 
special educators may now ask multi-dimensional questions that have long 
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perplexed and hindered Investigations and have contributed to the artificial 
'theory versus practice' polemic (Everitt, 1975). 



Rather than restricting the original evaluation question (s), the multivariate 
based model presented in this paper is designed to permit extending the 
number and types of questions that can be asked in an evaluation study 
through the application of multiple linear regression analysis. The planning 
and evaluation model is divided into four sections: (1) Specification of 
Program Objectives; (2) Pre Assessment Procedures; (3) Evaluation Methodology; 
and (4) Post Assessment Procedures. A description of the Intergal aspects 
of each model section follows; 
I. Specification of Program Objectives 

The most important step in planning any special education program is the 
accurate specification of the objectives. Before implementing the assessment 
of a special education program, it is essential to establish the extent of 
the need for such a program. Often this is a multi-phase procr s where 
initial surveys are made to determine what goals and objectives have been 
purported for the program and whether any problems exist. Since it is 
usually impractical to administer surveys tp^ derive such information from 
an entire population of exceptional children, sampling techniques and infer- 
ential statistics "are generally applied. If the pilot survey suggests that 
a problem exists, decision-makers must establish priorities for exploring 
the concomitants of the problem. It is at this point that relatively soph- 
isticated inferential models are essential. 

The objectives are first identified on the basis of a projection of the 
desired program outcomes. Such goals and objectives are defined through staff 
development and inservice education activities. To reduce the risk of a 
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pre-mature evaluation, it becomes necessary to collect baseline data from 
either or both of the following sources: 

(1) Previous program information and/or: 

(2-S Loneitudinal data gathered over a specified period of time on 

thofe objectives initially defined as relevant evaluative concerns. 

Quantifiable data is the only requirement that multiple linear regression 
analysis needs before inference is possible and many approaches to inferring 
the above are available. For example, one method may be to obtain the data in 
a post-hoc fashion by charting progress on one's defined objectives from an 
initial special education program (probably some years ago if the evaluation 
is of the longitudinal variety) to the most recent program. Depending upon 
your criterion (program accomplishments), the data gathered may be placed 
in a regression model allowing one to check for significant differences on 
various measures from one program year to another. This technique could be 
used as a measure of program, ob j active (s), and/or treatment success. 
Extending this technique, it would be possible to determine the most parsim- 
onious set of reasons responsible for the successes and failures of the desig- 
nated program. Technical operations would include the measurement of inter- 
action and possible curvilinear effects with thoughts of finding a systematic 
solution in the .determination of program accomplishments.. 

Another method would be the longitudinal analysis of the program variables. _ 
Independent and dependent variable fluctuations may be measured through the 
appUcatlon of. the multiple regression model. Once obtained, baseline data 
profiles are charted, plotting independent and dependent variable relationships 
on 'a measurement of change' basis over time. The longitudinal method is more 
highly recommended if economic and time factors are not of great Importance, sin' 
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the Initial definition and specification of the program objectives which 
would be known upon the first collection of data, would require systematic 
monltorlzatlon to remain relatively constant throughout an extended period of 
time. In retrospect, the post-hoc analysis leaves the evaluator speculating 
as to just exactly 'how the present program objectives differ In context 
from the Initial objectives.' Also, the question arises concerning whether 
or not the same evaluator was present at the Inception of the special educa- 
tion program (of a continuing series), and secondly, has his definition and 
subsequent data collection procedures remained constant? 

Success or failure In achieving the specified program objectives can be 
determined through a time-series analysis which compares each program 
objective from time 1 to time 2... .to the next time. Hultl-varlate analysis 
will assist In the diagnosis of possible resulting problems that would have 
prevented program needs from being met and opportunities from being used. 
This program development procedure provides an essential basis for developing 
evaluation objectives whose accomplishment will result In program Improvement. 

II. Pre Assessment Procedures 

Prior to the Implementation of a special program (or shortly after such a 
program has begun) a selected or randomly sampled segment of the group re- 
ceiving the program Is assessed to determine their present level' of function- 
Ing on the objectives. Initial steps taken during the preassessment phase 
include: (1) a review of proposed strategies by selected panels of special 
education specialists, (2) a review of on-going programs In oth^r geographic 
areas that may have relevance for meeting similar objectives, and; (3) the 
planning of specific procedures and time schedules to Implement the strategies 



When a ntmber of strategies are under consideration, a multl-varlate analysis 
can be used to assess the relative benefits of each. This Information, In 
conjunction with the data collected during the specification of objectives 
phase. Is used by the declslon-*inakers In selecting appropriate alternatives 
for given situations. 

Information Is then collected on statistically Inferred Independent and 
dependent relationships among the special education program variables. 
These data can then be used to answer questions of the following variety: 

(1) How much of what Is to be learned and demonstrated In the 
program Is already known by the sample (population tinder study)? 

(2) Does the sample have the prerequisite behavioral and physical 
capacity for the program they are receiving? 

The results of preassessment will provide information regarding: 

If and to what extent pre-program differences exist among group 

and Individual characteristics of the population who will be 
receiving the program. 

At the point where alternative strategies are Implemented, a procedure for 

monitoring them is used to Insure that they conform to the program design. 

By systematically comparing programs on a longitudinal basis, the effect 

of specific program alternatives for a given system can be well decumented. 

This type of information is absolutely essential to make valid decisions 

regarding the maintenance, modification, or dlscontlnixatlon of an ongoing 

program. The preassessment phase provides the program decision-makers 

with information about the present state of affairs surrotindlng the stability 

of objectives and the children receiving such program services. 
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Ill, Evaluation Methodology 

Multiple linear regression analysis can adequately reflect the complex- 
ities of the newer evaluation theories, consequently it was selected as 
the basic statistical methodology for the multi-variate based model. From 
research we know a multi-facet question requires a multi-facet method- 
ological approach before an appropriate solution can be found. For 
example, Sommers, Joiner and others (1970) speak of the multi-dimension- 
ality of research concerns when behavioral entities are involved: 

"It is assumed that performance or behavior is subject to the 
influence of more than one variable or condition at a time and that 
adequate explanations involve more than a single variable or condition. 
But, if several variables are proposed as being relevant to perform- 
ance it becomes necessairy to measure both the influence of the 
variables on the behavior we are attempting to explain and their 
Influence upon each other.'* 

The multi-variate procedure allows practitioners the freedom to apply a 
theoretical multi-behavior model to the real world of special education 
problem solving and gamer generalizable alternatives. It is accepted 
that people act in similar ways, but often for different reasons. In 
recent years we have recognized the problems inherent in assuming that 
a specific behavior has the same origin in all cases. It is within this 
framework that individualized approaches to education for children with 
problems in learning have developed. In this situation, a large number 
of devices may become important in aiding in the development of an • 
educational program for individual students* In addition to determining 
the level at which the student is functioning, it may be Important to 
identify particular areas of weakness or of strength which can be taken 
in account when planning. 
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As we develop more sophistication regarding the nature of the learning 
process of students In educational programs, the need for more precise 
assessment techniques and the corresponding remedial treatments become 
imperative. Multl-varlate analysis provides for a system to delineate 
the most efficient assessment techniques and the most effective program 
alternatives to hasten acceptance of the special education philosophy of 
styling programs after student characteristics and not vice versa. 

Multiple regression analysis, being an adaptable form of multl-varlate 
analysis provides the Investigative ability to statistical predication to 
assess operating systems In education. The power of prediction as an 
Intellectual tool resides In the fact that It enables one to rigorously 
test the adequacy of various theoretical evaluation models that might be 
proposed. In this procedure. It Is assumed that If certain variables predict 
events that are of Interest to one In a fairly accurate way, they are more 
relevant to the behavior or program objectlve(s) than those which do not. 
In terns of evaluation settings, variables which are highly predictive of 
attainment levels shotild be taken Into account In programming to a greater 
extent than those variables which are not. 

IV. Post Assessment Procedures 

Following the selection and preassessment of the program objectives, followed 
by the Instructional process. Implementation of the post assessment procedures 
are used to determine to what extent the objectives have been reached. 
After the program has been In operation for a pre-determlned length of 
time, the children who are receiving the program are assessed to determine 
whether differences exist between data collected In the preassessment phase 
as compared to data collected In the postassessment phase*. 
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Another consideration evidenced at this time Is the provision of total 
program Information to persons responsible for" Implementing plans and pro- 
cedures. After the results have been collected, an operational Information 
system would exhibit the following functions: 

(1) • • • to detect or predict defects In the procedural design or * 

It's implementation during implementation stages; 

(2) • • • to provide Information for programming decisions; and 

(3) • • • to maintain a record of the procedure as it occurs. 
The purpose of the Information system is to assure the provision that 
data is available for measurement and Interpretation of attainments not 
only at the end of a program's cycle, but as often as deemed necessary 
during each phase of the program. General considerations effecting the 
operation of a continually refining information system include the following: 

(1) • • • devising quantitative special education program objectives; 

(2) • • • measuring criteria associated with the objectives of the 

activity; 

(3) • • • comparing criteria success with predetermined absolute or 

relative standards; and 

(4) • • • generalizing Interpretable results of the outcomes using 

recorded specification, pre*»assessment, and post-*asses8ment 
information- 

This type of an information system would provide for the selection of one 
or a combination of the following alternatives in the change process: 

(1) • • • deciding to continue; 

(2) • • • terminate; 

(3) • • • modify or re-focus an activity. 

Multl-varlate analysis is applicable to any planning and evaluation set- 
ting in which it is possible to quantify your data. Webster and Elchel- 
berger (1970) have attempted to explain the utilization of multiple 
regression analysis, as generalized by Bottenberg and Ward (1960), 



Brodley (1968), Jennings (1967), and Kelly, Be ggs, McNeil, Eichelberger, 
and Lyon (1969) within the context of the Stuffelbeani (CIPP) evaluation 
model. The emphasis of their application was to promote the use of in- 
ferential statistics for the evaluation of educational programs in the public 
school environment* 

Much concern has been voiced about the failure of evaluation in special 

education. The efforts of Webster and Eichelberger appear to be the first 

in an attempt to become more precise with reference to the evaluation^ 

paradigm. The utilization of mxilti-variate analysis may be part of the 

solution to the failure of evaluation, Guba (1969), supporting what has 

been a general consensus of opinion by educational evaluators, examines 

the evaluation paradigm by focusing on certain lacks and submits the questions: 

"... why cannot evaluation programs be designed and implemented that 
will quidcly eradicate the following crucial lacks? 

1) . lack of adequate definition of evaluation 

2) lack of evaluation theory 

3) the inapplicability of assumptions 

4) the impossibility of continuous refinement 

5) lack of knowledge about decision processes 

6) lack of criteria" 

Guba concludes by stating: 

"The primary task in evaluation today is the provision of sensible alternatives 
to the evaluatot. The evaluation of educational innovations awaits modernization 
of the theory and practice of the evaluation art. We need then, a technology 
of evaluation." 

The crucial lacks of present planning and evaluation efforts present a challenge 
to evaluation designers. Multi-variate procedures appear to be an effective 
approach to assess the quantitative relationships between variables relevant 
to each of the categorical lacks designated by Guba. 
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SUMMARY 



Evaluation studies In special education have the common goal of producing 
planning and evaluative statements concerning the worth of particular 
educational activities and the relative progress of special education 
programming. The current status of special education planning and 
evaluation necessitates modem theory and methodology, since tradition- 
ally structured programs have failed special educators In their attempts 
to reflect and assess relevant educational concerns. What Is needed In 
special education, then Is a technology of evaluation allowing standard 
methods to be generalized In every evaluation setting. Once the evaluation 

process Is systematically structured Into a feasible operating system, 

■I 

the ensuing evaluation will reflect and maximize the efficiency with 
which program objectives are achieved. Upon that basis, an attempt has 
been made to coxistruct and exhibit the usefulness of a multivariate 
planning and evaluation model. 

The model which is technically based on multi-variate analysis is divided 
into four sections: (1) Specification of Program Objectives; (2) Pre 
Assessment Procedures: (3) Evaluation ^thodology; and (4) Post Assess- 
ment Procedures. Such a system would provide the necessary information 
required to appropriately plan and evaltiate special education programs. 
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